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"Pilot  workload  saturation  in  the  cockpit  of  US  Navy 
Aircraft  has  become  a  serious  concern.  Literature,  studies, 
and  flight  tests  indicate  that  utilizing  a  voice  interactive 
system  for  certain  cockpit  tasks  can  reduce  this  workload  by 
decreasing  the  time  required  to  perform  the  task. 

This  being  the  case,  the  problem  which  remains  is  one  of 
deciding  which  tasks  to  convert.  Therefore,  a  model  has 
been  developed  which  provides  the  designer  with  a  total 
systems  approach  for  use  in  deciding  what  combination  of 
tasks,  which  if  converted  for  performance  by  the  voice 
interactive  system,  will  result  in  the  greatest  workload 
reduction  without  overloading  the  pilot's  voice  channels. 
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I.  INTRODUCTION 


A.  BACKGROUND 

A  major  problem  for  the  U  S  Navy  flight  program  is  pilot 
workload  saturation  in  the  cockpit.  This  creates  a 
situation  in  which  there  is  no  reserve  to  deal  with 
unforeseen  contingencies  and  no  room  to  add  other  needed 
tasks.  The  Naval  Air  Development  Center  (NADC)  located  at 
Warminster,  Pennsylvania,  is  particularly  interested  in 
finding  a  solution  which  can  be  implemented  without  creating 
a  corresponding  degradation  in  the  performance  of  the 
individual  aircraft's  mission  (Interview  with  Warner,  1987]. 

Research  in  the  area  of  dual-task  performance  indicates 
four  ways  of  tackling  this  problem  [Wickens,1981] . 

1.  Training  pilots  to  perform  multiple  tasks  more 
efficiently. 

2.  The  use  of  criteria  whici»  select  those  individuals 
with  more  proficient  time-sharing  skills  in  the 
recruitment  of  pilots. 

3.  Designing  the  cockpits  so  that  the  pilot  will  only  be 
requested  to  concurrently  perform  tasks  which  are  more 
efficiently  shared. 

4.  Design  the  placement  of  controls  and  displays  in  the 
cockpit  to  allow  for  greatest  time-sharing  efficiency. 

The  design  of  the  cockpit  has  been  studied  extensively 
so  that  it  is  doubtful  that  any  further  large  reductions  in 
workload  can  be  achieved  in  that  area.  Pilot  selection 
could  not  be  based  solely  on  time-sharing  abilities  even  if 
techniques  for  accurate  measurement  of  this  ability  were 
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currently  available.  Pilot  selection  must  be  based  on  many 
more  important  criteria.  Also,  since  pilots  already  receive 
extensive  training  it  is  doubtful  that  any  significant 
increases  in  efficiency  can  be  achieved  by  providing 
additional  training.  Therefore,  attention  is  being  focused 
on  the  third  alternative  of  trying  to  ensure  that  tasks, 
which  a  pilot  must  perform  concurrently,  are  compatible  for 
the  purpose  of  efficient  time-sharing. 

Recent  studies  indicate  that  the  use  of  voice 
recognition/synthesis  in  an  aircraft  cockpit  may  provide  the 
best  means  of  achieving  the  compatibility  of  tasks  which 
will  achieve  the  desired  goal  of  workload  reduction. 

B.  LITERATURE  SEARCH 

A  large  volume  of  material  dealing  with  operator 
workload  has  been  generated  in  recent  years.  This  interest 
in  operator  workload  is  most  likely  the  result  of  the 
existence  of  jobs  which  require  multiple  tasks  to  be 
performed  in  a  very  complex  environment.  An  excellent 
example  of  such  a  job  is  the  piloting  of  a  U.  S.  Navy 
aircraft . 

Literature  generated  in  the  area  of  human  performance 
covers  all  aspects  of  time-sharing,  display  design,  single 
and  multiple  tasking,  attention  resources,  etc.  Because  of 
this  pro]  feration  of  research  it  is  necessary  to  limit  the 


review  of  this  materiel  to  that  of  attention  resources  and 
time-sharing.  These  two  areas  provide  the  theoretical  basis 
for  pursuing  the  use  of  voice  recognition/synthesis  in  an 
aircraft  cockpit  as  a  means  to  reduce  pilot  workload. 

1.  Attention  as  a  Limited  Resource 

In  1967  Moray  published  a  paper  which  highlighted 
the  quantitative  aspects  of  attention.  He  asserted  that 
attention  should  be  thought  of  as  having  a  limited  capacity 
in  much  the  same  way  that  a  given  computer  is  limited  in  how 
much  data  it  can  process.  Additionally,  he  stated  that  this 
capacity  could  be  divided  into  various  amounts  to  be 
assigned  to  tasks  on  the  basis  of  difficulty  and  demand. 
Particularly  beneficial  was  the  flexibility  and  ability  to 
share  these  resources  that  was  highlighted  in  his  treatment 
of  the  subject.  [Wickens,1984:pp.  292-293] 

2 .  Kahneman  Model 

The  Kahneman  model  is  important  because  it  was  the 
first  attempt  to  predict  performance  based  on  the  limited 
capacity/resource  theory.  However,  his  model  still  dealt 
with  attention  as  having  a  single  source  capacity  and  could 
not  account  for  instances  in  which  more  difficult  secondary 
tasks  interfered  less  with  the  primary  task  than  other  less 
difficult  tasks.  This  seemed  to  indicate  that  the  theory  of 
a  single  resource  was  possibly  too  simplistic. 

[ Wlckens,  1984  and  Wickens, 1987 ] 
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Multiple-Resource  Theory 


The  single  resource  theory  of  the  Kahneman  Model  was 
valuable  in  asserting  the  idea  of  attention  as  a  limited 
capacity  resource.  However,  it  failed  to  explain 
differences  from  predicted  results  which  occurred  when 
certain  tasks  were  combined.  (Wickens,  1964] 

in  an  attempt  to  resolve  these  apparent 
discrepancies,  Wickens  developed  a  Multiple-Resource  model 
which  combined  not  only  the  theory  of  attention  as  a 
resource  but  also  the  theories  of  attention  as  structural  in 
nature  (wickens, 1961,  Wickens, undated,  Wickens, 1966  and 
Wickens,  1964]. 

Building  on  research  by  individuals  such  as  Pachella 
(1974),  Kinsbourne  and  Hicks  (1978),  Harris,  Owens  and  North 
(1976)  and  others  [Wickens, undated ] ;  Wickens  designed  a 
three  dimensional  model  (Figure  1)  which  describes  attention 
along  the  dimensions  of  (1)  stages  of  processing,  (2) 
cerebral  hemisphere  of  operation  and  (3)  modalities  of 
processing  (encoding  and  response). 

Stages  of  processing  include  the  processes  of 
encoding,  memory  &  transformations,  and  responding.  The 
idea  of  different  sequentially  ordered  stages  of  processing 
has  achieved  a  certain  level  of  acceptance  and  even  though 
it  cannot  be  inferred  that  independent  resources  are  used. 
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some  research  In  dual  tasking  seems  to  Indicate  it  could  be. 
(Vickens,  undated] 

STAGES  OF  PROCESSING 


Figure  1.  Multiple  Resource  Model  T 

(Adapted  from  Wickens  (undated)) 

The  research  done  in  the  area  of  separate  cerebral 
hemispheres  of  operation  argues  strongly  that  the  two 
cerebral  hemispheres  act  as  separate  resource  reservoirs. 

In  particular,  spatial  tasks  along  with  left  hand  controls 
are  handled  by  the  right  hemisphere  of  the  brain  whereas 
verbal  tasks  and  right  hand  controls  are  handled  in  the 
left.  Since  a  slight  additional  amount  of  time  is  needed  in 
order  for  the  brain  to  send  messages  between  the  hemispheres 
of  the  brain,  it  is  considered  advantageous  in  certain 
circumstances  to  keep  processing  of  a  single  task  confined 
to  one  hemisphere.  However,  in  other  situations  it  is 
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beneficial  to  use  both  hemispheres  since  competition  for  the 
same  resources  can  cause  a  slower  response  than  that 
resulting  from  hemisphere  crossover  time. 

Some  of  the  clearest  measurements  have  been  along 
the  dimension  of  different  modalities  of  processing  in  which 
auditory  versus  visual  encoding,  and  manual  versus  vocal 
responses  have  been  studied.  Sandry  demonstrated  the  effect 
of  these  modalities  in  her  experiment  conducted  on  the  F-18 
simulator  located  at  the  Naval  Air  Test  Center,  Patuxent 
River,  Maryland  in  1981  (Sandry,1982 1 . 

4.  Sandry  Study 

In  her  testing,  Sandry  paired  verbal  and  spatial 
tasks1  with  the  continuous  primary  spatial  task  of  flying 
an  aircraft.  By  comparing  reaction  times  of  different 
input/output  modality  pairs,  inferences  concerning  expected 
performance  could  be  made.  In  general,  she  found  that  the 
time  required  to  complete  a  verbal  task  could  be  reduced  by 
changing  to  a  modality  combination  which  utilized  voice 
recognition  and/or  synthesis  while  such  a  combination 
resulted  in  degraded  spatial  task  performance. 


Wlckens  defines  verbal  tasks  as  "Tasks  for  which  words, 
language,  or  logical  operations  are  natural  mediators  ...". 
Spatial  tasks  are  defined  as  "...  almost  any  task  that  requires 
moving,  positioning,  or  orienting  objects  in  space,  or  performing 
other  analog  transformations,..."  [Wickens,1987:p.  161. 
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sandry's  study  was  useful  in  two  ways.  First,  she 
directs  attention  immediately  to  those  tasks  which  will  be 
most  benefited  by  voice  implementation.  Second,  because  the 
data  was  gathered  on  the  Navy's  F-18  simulator,  it  can  be 
used  for  an  initial  cut  in  projecting  possible  reaction  time 
savings . 

C.  DEPARTMENT  OF  DEFENSE  STUDIES 

1.  Air  Force  B-52  Bomber  Study  [North  and  Lea, 19821 
This  study  represents  the  armed  forces'  first 

attempt  to  identify  cockpit  tasks  which  would  be  benefited 
most  by  conversion  from  visual/manual  input/output 
modalities  to  audlo/vocal  modalities.  The  approach 
developed  by  Honeywell  dealt  with  cockpit  tasks  on  an 
individual  mission  basis.  A  filtering  system  was  then 
developed  which  screened  and  prioritized  candidate  tasks 
using  a  series  of  objective  and  subjective  criteria. 

2.  Navy  F/A-18  Study  [Mountford,  North,  Metz,  and 

Graffunder, 19821 

The  Navy  study,  which  was  also  conducted  by 
Honeywell,  had  three  objectives.  They  were  to  (1)  perform  a 
literature  review,  (2)  develop  task  selection  procedures, 
and  (3)  study  'dialogue'  issues.  Of  primary  interest  to 
this  thesis  is  the  second  objective  of  task  selection.  The 
resulting  procedures  refined  those  developed  as  part  of  the 
B-52  study. 
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These  two  studies  performed  by  Honeywell,  although 
valuable,  are  weak  in  two  areas.  First,  determination  of 
technological  feasibility  for  a  particular  candidate  task 
for  voice  implementation  is  done  with  a  weighting  factor. 

It  would  be  much  more  straight  forward  to  use  a  go/no-go 
decision  process  for  this.  When  Warner  at  NADC  attempted  to 
use  the  proposed  filtering  process,  tasks  which  were  not 
technologically  feasible  were  not  successfully  screened  out 
[Interview  With  Warner, 1987 ) . 

Secondly,  the  filtering  process  designed  during 
these  studies  chooses  and  prioritizes  candidate  tasks  on  an 
individual  basis  with  only  a  limited  ability  to  judge  the 
interference  effect  when  multiple  tasks  are  converted  to 
voice.  There  is  no  attempt  to  determine  the  most 
advantageous  combination  of  voice  tasks. 

D.  DEPARTMENT  OF  DEFENSE  FLIGHT  TESTS 

The  military  services,  in  spite  of  some  valid 
reservations,  have  displayed  a  continued  interest  in  voice 
synthesis/recognition  as  a  possible  solution  for  the 
saturated  workload  situation  present  in  the  aircraft 
cockpit.  As  a  result,  flight  tests  have  been  performed,  the 
results  of  which  are  discussed  below. 


1.  F/A-16A  Flight  Tests  ILoikith  and  Hall, 19863 


The  purpose  of  these  flight  tests  was  to  determine 
the  feasibility  and  pilot  utility  of  voice 
synthesis/recognition.  Results  of  the  tests  indicated  that 
voice  implementation  of  cockpit  tasks  was  feasible  and  could 
be  useful  provided  that  technical  problems  with  the 
particular  voice  system  tested  could  be  corrected.  The  Lear 
Siegler,  Inc;  Voice  Control  Interactive  Device  was  used  for 
these  tests. 

2.  AFTI/F-16  Flight  Tests  (Williamson, undated] 

Using  the  Texas  Instrument  Voice  Interactive  System 
the  Air  Force  obtained  impressive  results.  The  following 
excerpt  from  the  "Discussion"  portion  of  the  report  reveals 
how  well  this  testing  went. 

The  test  pilots  found  it  safer  and  more  convenient  to  use 
voice  instead  of  traversing  the  awkward  menu  logic  of 
multifunction  keyboards  and  displays.  The  ability  to 
request  information  verbally  and  receive  a  verbal  response 
has  also  received  universal  praise  from  the  pilots. 

This  report  also  highlights  the  fact  that 
development  of  a  method  for  selecting  the  most  promising 
tasks  for  conversion  to  voice  is  necessary. 

3.  JOH-58C  Helicopter  Flight  Tests  ( Szerszynski , 1987 ] 

The  Army  also  experienced  favorable  results  using 

voice  in  the  JOH-58C  helicopter.  One  of  the  sub-tests 
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conducted  was  a  simulated  single  pilot  missior^  during 


which  the  pilot  had  to  simultaneously  control  flight  tasks, 
radio  control,  and  visual  search  .  Their  assessment  of  the 
test  results  was  that  "The  voice  controlled  avionics  system 
is,  based  on  the  data,  the  avionics  control  system  of  choice 
when  the  workload  is  high". 

These  tests  all  demonstrate  the  ability  of  voice 
synthesis/recognition  as  a  means  to  reduce  pilot  workload 
thereby  increasing  efficiency  and  safety.  The  last  two 
tests  in  particular  confirm  the  fact  that  the  necessary 
voice  technology  is  currently  in  existence. 


‘The  phrase  "simulated  single  pilot  mission"  was  used  since 
second  pilot  was  along  for  safety  reasons. 


II .  MODEL 


A.  OVERVIEW 

The  primary  goal  of  this  thesis  is  to  combine  previous 
research  and  DOD  studies  into  a  single  model  which  will 
provide  the  design  engineer  with  guidelines  and  procedures 
for  a  total  systems  approach  to  implementing  voice 
recognition/synthesis  in  the  cockpit.  Wickens  states  the 
goal  of  this  model  best. 

A  major  goal  of  predictive  performance  models  is  to  help 
determine  the  most  effective  design  before  a  system  is 
configured,  such  models  are  not  likely  to  be  100% 
reliable.  However,  they  may  prevent  the  designer  from 
developing  prototypes  that  will  be  clearly  non-optimal, 
and  will  also  enable  the  designer  to  identify  a  parameter 
space  that  is  most  worthy  of  experimental  investigation. 
[Wickens, 1988 ] 

Likewise,  this  model  will  provide  the  most  beneficial 
starting  place  for  design  and  testing  but  cannot  be  expected 
to  provide  "the  answer".  Variances  between  aircraft  and 
ever  changing  missions  preclude  any  other  approach. 

This  model  draws  extensively  from  the  methods  developed 
in  the  Honeywell  studies  [North  and  Lea, 1982  and  Mountford, 
North,  Metz  and  Graf funder, 1982 ]  which  use  a  total  systems 
approach  in  deciding  what  systems  to  activate  by  voice.  The 
performance  data  generated  by  Sandry  (1982)  provides  the 
data  for  use  in  projecting  expected  workload  savings.  The 
product  of  this  model  will  allow  the  design  engineer  to 
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begin  with  a  first  cut  will  maximize  workload  savings  while 
avoiding  overloading  the  pilot  on  the  voice  interactive 
side . 


To  aid  in  the  future  refinement  of  this  model, 
individual  modules  will  be  used  to  separate  distinct 
portions.  Detailed  descriptions  of  these  modules  follow. 

B.  MODEL  DESCRIPTION 
1.  Module  One 

a.  Task  Analysis: 

This  portion  of  the  model  was  developed  by 
Honeywell  [North  and  Lea,1982:pp.  8-14  and  Mountford,  North, 
Metz  and  Graf funder, 1982:pp.  6-18]  as  part  of  two  studies 
performed  for  NADC. 

(1)  Mission  Scenario:  According  to  the 
Honeywell  studies,  the  first  step  in  analyzing  an  aircraft 
cockpit  regarding  the  value  of  implementing  voice 
synthesis/recognition  is  to  develop  a  mission  scenario. 

This  scenario  should  be  a  general  narrative  which  accurately 
describes  the  mission  under  consideration. 

Honeywell  deals  with  mission  scenarios 
individually,  taking  each  mission  separately  and  running  it 
through  the  entire  procedure.  In  this  model  a  broader  look 
is  taken.  All  of  the  major  missions  of  the  particular 
aircraft  under  study  should  be  developed  and  evaluated 


together  to  provide  an  analysis  which  produces 
recommendations  for  voice  implementation  based  on  the 
aircraft  as  a  total  system. 


In  choosing  the  missions  to  evaluate, 
consideration  should  be  given  to  high  workload  missions 
although  it  is  important  to  include  all  major  missions. 
Infrequently  flown  missions  should  not  be  considered  unless 
unusually  critical  since  they  will  unduly  affect  the 
findings  of  the  evaluation  process. 

(2)  Task  Narrative:  The  second  step  advocated 
by  Honeywell,  is  the  development  of  a  task  narrative  which 
provides  a  detailed  verbal  description  of  each  task  of  the 
mission.  A  task  narrative  should  be  developed  for  each 
mission  scenario.  An  example  of  a  task  narrative  is 
contained  in  the  F-18  Honeywell  study  [North  and  Lea,1982:p 
9]  and  has  been  provided  as  Table  1.  The  tasks  should  be 
listed  in  the  order  in  which  they  occur  during  the  flight. 

(3)  Time-based  Activity  Loo:  The  next  step  in 
the  Honeywell  procedure  is  to  use  the  Task  Narrative  to 
"...categorize  each  activity  into  its  human  information 
processing  channels--vocal,  visual,  auditory,  and  manual." 
[Mountford,  North,  Metz,  and  Graf funder, 1982 :p.  10],  Also 
an  index  should  be  included  which  indicates  the  difficulty 
in  distance  and  location  the  pilot  experiences  in  reaching 
for  the  manual  control.  Those  tasks  for  which  the  pilot 


does  not  reach  would  be  assigned  a  value  o£  zero  and 
Increase  In  Incremental  steps  of  one  up  to  the  most 
difficult.  This  reach  index  is  a  useful  measure  when 
combined  with  other  information  in  deciding  the  best 
candidates  for  speech  synthesis/recognition.  In  Honeywell's 
F-18  study  example  the  highest  value  was  a  three.  Figure  2 
provides  an  example  of  hew  this  index  could  be  assigned  to 
an  aircraft  cockpit. 


Table  1.  TASK  NARRATIVE 
(Adapted  From  North  and  Lea  (1982)) 


Pilot  Tasks 

Copilot  Tasks 

Additional  Aircrew  Tasks 

Calls  for  preparation  for 

Reads  preparation  for 

RN — Cells  renne  to  tanker  In 

contact  checklist. 

contact  checklist. 

1  NM  Increments.  Calls 

1.  establish  radio  contact 

2.  Check  etr  conditioning 
system. 

range  at  3  ...  2  m. 

6.  Disengage  autopilot. 

4.  Turn  antlcollJsion  lights 
orr. 

5.  Turn  nevlqetion 
lights  to  flash. 

6.  Set  slipway  and  airplane 

N— 3.  Set  rendezvous  radar 
to  standby. 

7.  Set  airbrake  lever 

light  switches  to  full 
bright. 

7,  Open  slipway  doors  snd 

to  position  1. 

verify  ready  lights  on. 

8.  Determine  tanker  position 

RN — 8.  Determine  tanker 

on  FLIR  sensor. 

position  on  FVIR 

9.  Select  FUR  video. 

10.  Set  air  refueling 
switch  to  ai r  refuel . 

sensor. 
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Figure  2.  Anthropometric  Zones 
(Adapted  From  North  and  Lea(1982)) 


An  example  of  the  Time-based  Activity  Log  is 
included  as  Table  2  (Mountford,  North,  Metz,  and 
Graf funder, 1982:p.  11),  Immediately  obvious  in  this  sample 
is  the  fact  that  many  of  the  tasks  show  input  or  output 
modes  of  "N"  (not  applicable).  This  is  not  appropriate. 

Bach  task  can  have  both  input  and  output  modes  specifically 
identified.  For  the  case  where  a  Mnot  applicable"  entry 
appears  appropriate  the  following  guidelines  should  be 
employed.  If  the  task  is  prompted  by  a  checklist  it  should 
be  considered  to  have  a  visual  input  mode.  If  it  is  a 
memory  item,  then  it  should  be  listed  as  an  audio  input  mode 
because  this  most  closely  identifies  the  information 
processing  channel  (hemisphere)  used.  In  the  case  of  output 
modes  the  vocal  mode  should  be  identified.  This  follows 
from  the  fact  that  if  no  obvious  manual  action  occurs  then 
the  information  processing  channels  activated  must  be  the 
same  as  those  used  for  a  vocal  response. 

(4)  Activity  Chart:  This  chart  is  the  graphic 
form  of  time-based  activity  log.  See  Figure  3  below 
(Mountford,  North,  hetzr  and  Graf funder, 1982 :p.  13).  These 
charts  are  particularly  important  to  the  subsequent 
screening  of  proposed  tasks  for  voice  Implementation  since 
they  provide  quick  information  on  which  tasks  are  performed 
in  conjunction  with  the  flight  task  thereby  creating  a  dual 


Table  2.  TIME-BASED  ACTIVITY  LOG 
(Adapted  from  Mountford,  Narth,  Metz,  and  Graffunder  (1982)) 


tasking  situation.  This  information  will  bo  necessary 
during  the  optimization  portion  of  this  model. 


mar 

•aunt 


tainw 


MftltT 

Hum 


Figure  3.  Activity  Chart 

(Adapted  From  Mountford,  North,  Metz,  and  Graf funder  (1982) 

b.  Development  of  Speech  I/O  Candidate  Task  List: 
After  producing  the  task  analysis  for  all 
pertinent  scenarios,  an  initial  list  of  potential 
candidates  for  conversion  to  voice  should  be  compiled. 

Bach  task  should  be  evaluated  by  the  following  series  of 
filters.  These  filters  provide  the  first  gross  look  at  a 
given  task  to  judge  its  suitability  for  voice. 
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(1)  Verbal  Task  Filter;  This  first  filter  asks 


whether  or  not  the  task  can  be  classified  as  one  using  the 
verbal  information  processing  channels  (see  Footnote  1).  If 
the  answer  is  yes,  it  will  continue  through  the  filters 
which  follow.  Xf  no,  it  will  be  rejected  immediately. 
Currently,  only  verbal  tasks  benefit  from  conversion  to 
voice  synthesis/recognition. 

(2)  Speech  Recognition  Filter:  This  filter  is 
used  to  decide  whether  a  task  would  be  a  good  candidate  for 
speech  recognition.  The  questions  for  this  filter  were 
developed  in  the  Honeywell  B-52  study,  further  refined  in 
the  Honeywell  F-18  study  and  are  provided  in  flowchart  form 
as  Figure  4  (Hountford,  North,  Metz,  and  Graffunder,1982:p. 
201. 

One  additional  question  has  been  added  to 
this  filter.  The  first  question  asks  if  the  task  is 
currently  accomplished  with  a  by  pilot  speech.  If  the 
answer  is  yes,  it  will  be  immediately  rejected  since  it 
already  uses  voice  output.  If  no,  it  will  continue  through 
this  filter. 

The  question  concerning  the  number  of  times 
the  task  is  performed  during  the  mission  has  been 
transferred  to  its  own  filter  so  that  this  it  can  be  more 
efficiently  handled. 


19 


MAC1  TASK 


Figure  4.  Speech  Recognition  Filter 
(Adapted  From  Mountford,  North,  Metz,  and 
Graf funder  (1982)  and  Modified  By  Author) 


(3)  Speech  Generation(Synthesls)  Filter:  This 
filter  is  used  to  decide  whether  a  task  would  be  a  good 
candidate  for  speech  synthesis.  A  flowchart  for  the 
following  questions  is  provided  in  the  Honeywell  B-52  study 
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[North  and  Lea,1962:p.  21].  it  should  be  noted  that 
question  one  was  added  to  the  filter  by  this  author. 

1.  Is  the  task  currently  handled  by  speech  input  to  the 
pilot?  If  yes,  reject  task.  If  no,  continue  to 
evaluate . 

2.  Can  information  be  transmitted  to  pilot/co-pilot  as  a 
short  phrase?  If  no,  reject  task.  If  yes,  continue 
to  evaluate. 

3.  Does  it  occur  during  visual  time-sharing?  If  yes, 
accept  task.  If  no,  continue  to  evaluate. 

4.  Can  it  be  used  as  an  Information  display  that 
currently  does  not  occur  in  the  central  anthropometric 
visual  zone?  If  yes,  accept  task.  If  no,  continue  to 
evaluate . 

5.  Can  it  be  used  together  with  speech  recognition  in  an 
interactive  dialogue  format?  If  yes,  accept.  If  no, 
reject. 

(4)  Technical  Feasibility:  This  filter  is  used 
to  decide  whether  a  task  can  be  implemented  as  part  of  the 
voice  system  from  a  technical  standpoint  (e.g.,  if  a  task  is 
using  equipment  not  controlled  through  the  multi-plexer 
(mux),  the  voice  system  cannot  access  it  and  the  task  wiJl 
be  dropped  from  the  candidate  task  list).  This  is  a 
question  which  must  have  a  yes  or  no  answer  from  an 
engineering  standpoint.  Therefore,  each  task  which  was 
accepted  by  either  the  Speech  Recognition  Filter  and/or  the 
Speech  Generation  Filter  should  be  evaluated  technically. 

This  procedure  differs  from  the  Honeywell 
studies  which  address  this  issue  from  the  standpoint  of  a 
feasibility  scale  which  is  judged  to  provide  little 
additional  information  and  only  serves  to  obscure  the 
analysis  by  adding  an  additional  factor.  In  fact,  the  use 
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of  such  a  scale  could  lead  to  accepting  a  task  which  cannot 
be  implemented. 

■\>  (5)  Repetition  Filter:  At  this  point,  the  tasks 

which/ have  successfully  passed  through  the  previous  filters 
should  be  grouped  according  to  equipment  used,  input/output 
mode,  number  of  steps  to  complete,  and  anthropometry  index. 

This  is  done  so  that  the  frequency  with 
which  the  same  action's  are  taxen  can  be  judged*.  Once  this 
grouping  has  been  compiled,  the  decision  maker  must  decide 
how  many  times  a  task  must  be  repeated  before  it  becomes 
advantageous  to  consider  conversion  to  voice  modes. 

Once  this  decision  has  been  made,  only  those 
tasks  which  occur  with  the  proper  frequency  will  be  placed 
on  the  Candidate  Task  List. 

(6)  Candidate  Task  List:  Once  a  task  has  been 
accepted  as  a  candidate  for  voice  synthesis  and/or 
recognition  it  should  be  placed  on  the  Candidate  Task  List 
which  is  the  output  for  this  module.  Along  with  each  task 
the  current  input/output  modes  and  the  anthropometry  index 
should  be  listed.  An  example  of  the  procedure  described  in 
this  module  is  included  in  the  Appendix. 

3This  handling  of  task  repetition  differs  from  the  Honeywell 
studies  in  that  instead  of  being  part  of  the  Speech  Recognition 
Filter,  it  is  handled  as  a  separate  filter.  The  reason  for  this 
change  is  that  grouping  the  tasks  as  described  above  prior  to 
asking  this  question  simplifies  the  process  considerably  when 
this  procedure  is  put  into  practice. 


2.  Module  Two — Pilot  Input 

Pilot  involvement  in  the  task  selection  process  is 
considered  essential  by  NADC  [Interview  With  Warner ,  1987]. 
Pilots  possess  a  wealth  of  knowledge  and  experience,  which 
if  properly  used,  will  greatly  enhance  this  selection 
process.  In  addition,  the  involvement  of  the  pilots  should 
result  in  greater  acceptance  of  this  new  technology. 

Both  of  the  Honeywell  Studies  [North  and  Lea, 1982 
and  Mountfora,  North,  Metz,  and  Graf funder,  1982]  address 
the  assignment  of  a  pilot  utility  rating  to  those  tasks 
which  are  contained  in  the  Candidate  Task  List  produced  in 
module  one  above . 

In  the  Honeywell  F/A-18  study  [Mountford,  North, 
Metz,  and  Graf funder,  19821,  it  was  suggested  that  pilots  be 
given  a  questionnaire  which  contains  questions  requiring 
checks  in  blocks  which  are  scaled  on  an  ordinal  basis  from 
one  (never  useful)  to  six  (always  very  useful).  The 
following  example  is  taken  directly  from  that  study. 

Example:  "Descent  Checklist” 

1  2  3  4  5 

Never 

Useful  _ 

Sometimes 
Useful 

Responses  should  be  compiled  to  show  both  how  many 


Always  very 
Useful 


pilots  selected  a  given  rating  and  the  average  rating.  This 


allows  the  decision  maker  to  choose  either  the  average  or 
that  rating  that  was  picked  most  often  (the  mode)  as  the 
combined  utility  rating  to  be  used  in  the  optimization 
module.  The  following  example  illustrates  this  process. 

Example:  "Descent  Checklist" 

1  2  3  4  5  6 

Never  Always  very  Average 

Useful _ 1 _ _J _ 8 _  )5»  12  Useful  4.83 

Sometimes 

Useful 

*IndJ cates  mode 

In  this  example,  there  is  very  little  difference 
between  the  average  and  the  mode  even  though  these  two 
figures  could  vary  greatly. 

In  evaluating  the  feedback  provided  by  this 
questionnaire,  strong  consideration  should  be  given  to 
deleting  any  task  from  the  candidate  list  which  receives 
ratings  which  fall  primarily  in  the  1  and  2  range.  If  not 
completely  eliminated,  these  tasks  could  still  be  selected 
in  the  optimization  module  if  other  weighting  factors  are 
high  enough.  In  most  cases  it  would  be  counter  productive 
to  provide  a  voice  task  which  the  pilots  have  already 
indicated  would  not  be  useful. 

The  output  of  this  module  is  a  list  of  candidate 
tasks  along  with  the  assigned  pilot  utility  rating  for  each. 
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.  Module  Three — Performance  Specifications 


The  goal  of  this  portion  of  the  model  is  to 
establish  performance  specifications  which  can  be  used  in 
Module  Four  to  maximize  workload  reduction  (reaction  time 
savings ) . 

These  performance  specifications  must  include  both 
the  single  and  dual  task  conditions.  The  reason  for 
inclusion  of  the  single  task  condition  is  that  the  mission's 
activity  chart  produced  in  module  one  includes  time  frames 
in  which  the  pilot/co-pilot  is  not  engaged  with  the  flight 
task,  thus  becoming  a  single  task  situation. 

In  establishing  performance  specifications,  it  is 
first  necessary  to  determine  whether  the  numbers  should 
reflect  the  worst  case,  best  case  or  expected  (average) 
case.  This  must  be  decided  on  an  individual  case  basis. 

One  method  for  establishing  these  specifications  is 
demonstrated  using  the  data  from  the  Sandry  study 
(Sandry, 1982 ] .  As  previously  mentioned,  reaction  times  and 
root  mean  squared  error  rates  were  gathered  through  use  of 
the  Navy's  F-18  simulator.  The  pilots  used  were  from  the 
local  flying  club  and  had  an  average  of  nine  years 
experience* . 


*  Since  F-18  pilots  are  used  for  data  gathering  efforts  only 
on  advanced  high  level  projects,  the  experience  level  of  the 
pilots  used  by  Sandry  was  appropriate. 


For  the  purposes  of  this  model,  workload  is  defined 
by  reaction  time.  This  means  that  the  reduction  of  the 
amount  of  time  necessary  to  complete  a  task  directly 
translates  into  a  reduction  of  workload  (reaction  time 
savings)  for  the  pilot/co-pilot. 

Prior  to  using  the  data  collected  by  Sandry,  its 
validity  for  the  purpose  of  establishing  performance 
specifications  was  evaluated.  A  linear  regression  model  was 
used  tor  this  analysis.  Using  the  St^tgraphics  Package  by 
STSC,  regressions  were  run  for  single  verbal  task  reaction 
times,  single  spatial  task  reaction  times,  dual  verbal  task 
reaction  times,  dual  spatial  task  reaction  times,  dual 
verbal  task  root  mean  square  values,  and  dual  spatial  task 
root  mean  square  values.  The  results  were  as  follows: 

1.  Residuals  indicated  that  the  linear  regression  model 
was  appropriate  for  the  reaction  times  but  not  the 
best  choice  for  root  mean  square  error  rates  (Figure 
5).  However,  since  it  is  only  necessary  to  judge 
general  trends  for  the  error  rate  (which  were 
adequately  identified  by  the  linear  regression),  no 
additional  models  were  evaluated. 

2.  Verbal  tasks  but  not  spatial  tasks  were  benefited  by 
conversion  to  voice  input  and/or  output  modes. 
Therefore,  no  performance  specifications  will  be 
developed  for  spatial  tasks. 

3.  Error  rate  declines  when  voice  input  and/or  output 
modes  are  used  (Figure  6). 

4.  In  a  single  task  condition,  a  variance  is  found  among 
the  subjects,  which  although  is  statistically 
significant,  disappears  in  the  dual  task  condition. 

5.  The  independence  of  the  variables  was  confirmed. 

In  establishing  the  performance  specifications  for 
this  demonstration,  mean  reaction  times  were  used. 


» I NOLI  TASK 

NORMAL  PROBABILITY  PLOT 


DUAL  TASK 

NORMAL  PROBABILITY  PLOT 


ROOT  MSAN  SQUARE 
NORMAL  PROBABILITY  PLOT 


-7  13  33  33  73 

Residuals 


Figure  5.  Normal  Probability  Plots  [in  secs] 


COMPONENT+RESIDUAL  PLOT 
SINGLE  TASK  REACTION  TIME 


V/M  V/V  A/M  A/V 

Input/Output  Mode 


COMPONENT+RESIDUAL  PLOT 
DUAL  TASK  REACTION  TIME 


V/M  V/V  A/M  A/V 


Input/Output  Mode 


Figure  6.  Component+Residual  Plots  tin  secs] 


single  and  Dual  Task  Input/Output  Modes 


Figure  7.  Plot  of  Reaction  Time  Variance^ 


5The  letters  S  and  D  were  used  to  distinguish  between  single 
and  dual  tasks,  respectively.  The  numbers  shown  with  these 
letters  Identify  the  input/output  modes  as  follows: 

1  *  Visual/Manual 

2  =  Visual/Vocal 

3  =  Audio/Manual 

4  =  Audio/ Vocal 
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Figure  7  shows  the  spread  of  the  reaction  times  by 
input/output  mode  and  by  single/dual  task  condition.  The 
trend  present  in  the  intervals  shown  is  consistent  with  that 
shown  by  the  means.  Therefore,  the  means  can  be  used  for 
ease  of  computation  purposes  without  distorting  the  outcome. 
The  increase  in  the  variance  between  the  single  and  dual 
task  conditions  is  explained  by  the  resource  capacity  theory 
discussed  in  Chapter  I.  The  variance  present,  while  a 
consideration,  does  not  prohibit  the  use  of  the  average  in 
establishing  performance  standards.  Table  3  below  provides 
the  performance  specifications  expected  for  tasks  in  each  of 
the  four  combinations  of  input/output  modes. 


Table  3.  AVERAGE  REACTION  TIMES 


Input/Qutput  Mode 
Visual/Manual  (V/M) 
Visual/Vocal  (v/v) 
Audio/Manual  (A/M) 
Audio/ Vocal  (A/V) 


Single  Task 
Reaction  Times 

10.03  [secs] 
8.37 
8.57 
6.8 


Dual  Task 
Reaction  Times 

17.91  [secs] 
15.69 
15.24 
11.59 
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Tables  4  and  5  provide  the  expected  difference  in 
reaction  time  which  will  occur  if  the  input/output  modality 
is  changed. 


Table  4. 

SINGLE  TASK 

REACTION 

TIME 

SAVINGS  (seconds 

V/M 

V/M 

0 

To 

V/V 

1.66 

A/M 

1.46 

A/V 

3.23 

From 

V/V 

-1.66 

0 

0.20 

1.57 

A/M 

-1.46 

0.20 

0 

1.77 

A/V 

-3.23 

-1.57  - 

1.77 

0 

Table  5.  DUAL  TASK 

REACTION  TIME 

SAVINGS  (seconds) 

V/M 

V/V 

To 

A/M 

A/V 

V/M 

0 

2.22 

3.67 

6.32 

V/V 

-2.22 

0 

0.45 

4.10 

A/M 

-3.67 

-0.45 

0 

6.32 

A/V 

-6.32 

-4.10 

-6.32 

0 

The  potential  for  reducing  reaction  time  is  found  by 
locating  the  row  which  lists  the  input/output  mode 
combination  currently  in  use  and  combining  this  with  the 
column  indicating  the  candidate  input/output  mode. 

In  many  cases  it  will  not  be  possible  to  pull  the 
numbers  from  these  tables  and  immediately  use  them.  It  is 
first  necessary  to  relate  the  task  at  hand  to  those 
performed  in  the  Sandry  study.  The  most  straight  forward 
way  to  accomplish  this  is  to  consider  the  number  of  steps 
necessary  to  perform  the  task  in  question.  The  Sandry  tasks 
required  four  steps  for  completion.  Therefore,  the  reaction  t 
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time  for  the  task  under  consideration  can  be  obtained  by 
multiplying  the  tabled  reaction  time  by  the  ratio  calculated 
by  dividing  the  number  of  steps  necessary  to  complete  the 
current  task  by  four. 

Tables  3,  4,  and  5  represent  the  type  of  output 
expected  from  this  module. 

4.  Module  4 — Optimization 

This  portion  cf  the  model  represents  the  heart  of 

the  total  systems  approach  to  implementation  of  voice 

synthesis/recognition  in  the  cockpit.  The  goal  of  this 

module  is  to  select  that  combination  of  candidate  tasks  with 

associated  input/output  modes  which  will  provide  the  largest 

reduction  in  workload  without  overloading  the  voice  channels 

used  by  the  voice  input/output  modes.  * 

Linear  programming  is  proposed  as  the  preferred 

method  for  accomplishing  this  objective.  This  method  was 

developed  as  a  means  of  dealing  with  situations  containing  a 

large  number  of  variables  with  associated  constraints.  As 

stated  by  the  Operations  Analysis  Group  (1984), 

Problems  of  allocation  arise  whenever  there  are  a  number 
of  activities  to  be  performed  but  limitations  on  either 
the  amount  of  resources  or  the  manner  in  which  they  may  be 
allocated  prevents  accomplishment  of  each  separate 
activity  in  the  most  effective  way  conceivable .. .a 
powerful  technique  that  has  been  developed  to  solve  such 
problems  is  called  mathematical  programming.  Vhen  the 
problem  can  be  formulated  within  a  mathematical  framework 
in  such  a  way  that  it  becomes  one  of  maximizing  or 
minimizing  a  linear  expression  subject  to  certain  linear 
constraints,  the  technique  is  known  as  linear  programming. 
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Likewise,  this  is  an  allocation  problem  and  it  can 
be  formulated  within  the  "mathematical  framework"  of  a 
linear  function  with  linear  constraints.  Such  a 


mathematical  formulation  follows. 


Maximize Lj  Lj  L|  Tjju 

such  that 

Ej  Rim 

xl)ll 


xmi 

(2-1) 

A 

■ 

_0 

(2-2) 

<-  1 

(2-3) 

^J»l  “  xi|t2  "  •••  "  xiJI»  (2“4) 

where 

i  »  Candidate  Task 
j  «  Mission 
k  »  Input/Output  Mode 

1  *  Type  of  Task  (single  or  dual  condition) 
Tjjn  »  Utility  Value  '  Of  Task  i.  Mission  j, 

Input/Output  Mode  k,  and  Type  of  Task  1 
»  The  Variable  of  Choice 
^jkl  *  Reaction  Time7  For  Task  1,  Mission  j, 

Input/Output  Mode  k,  and  Type  of  Task  1 
Cj  »  Total  Time  (in  seconds)  of  Additional 
Time  Allowed  For  Voice  Modes 
n  ■  Number  of  Missions  Under  Consideration 


The  objective  equation  in  (2-1)  maximizes  the  total  reaction 
time  savings  for  all  of  the  missions  under  consideration. 

The  constraint  equation  (2-2)  restricts  the  amount  of  time 
which  can  be  added  to  the  voice  information  processing 
chanrels  during  a  given  mission  so  that  the  pilot  will  not 


*The  utility  value  is  the  weighted  combination  of  reaction 
time  savings,  pilot  utility  and  anthropometry  index.  The 
Importance  of  each  element  is  dependent  upon  the  judgement  of  the 
decision  maker.  One  possible  weighting  scheme  is  demonstrated  in 
the  Appendix. 

7This  figure  is  obtained  from  Tables  4  and  5  as  appropriate. 
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become  overloaded  In  that  area.  This  value  Is  at  the 
discretion  of  the  decision  maker,  constraint  equation  (2-3) 
limits  the  selection  of  the  input/output  mode  to  only  one 
for  each  task.  Lastly,  constraint  equation  (2-4)  insures 
that  the  same  combination  of  tasks  will  be  selected  for  each 
mission.  This  constraint  Is  deleted  when  the  missions  do 
not  contain  Identical  tasks.  When  this  occurs  it  will  be 
necessary  to  evaluate  the  missions  separately  to  establish 
the  proper  mix  of  tasks.  However,  in  general,  the  same 
tasks  will  be  present  in  every  mission. 

Any  good  linear  programming  package  will  be  able  to 
solve  the  above  maximization  problem,  thereby  supplying  the 
system  designer  with  the  best  initial  combination  of  tasks. 
This  combination  of  tasks  will  represent  the  largest 
workload  savings  possible  while  not  overloading  the  pilots 
voice  interactive  channels.  The  LINDO  (Linear,  INteractive, 
Discrete  Optimizer)  software  package  by  LINDO,  Inc  is  used 
in  the  Appendix  to  solve  an  example  problem. 

C.  MODEL  SUMMARY 

This  model  has  Intentionally  been  designed  to  be  general 
in  nature  so  that  it  is  easily  applied  to  any  aircraft 
cockpit.  The  procedures  described  are  specific  enough  to 
guide  the  system  designer  while  retaining  the  flexibility 
necessary  to  address  the  needs  of  the  Individual  aircraft. 
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in  summary,  this  model  consists  o£  a  series  of  questions 
which  evaluate  the  potential  of  all  tasks  performed  during 
all  of  the  aircraft*s  missions.  Those  tasks  which  are 
considered  as  good  candidates  for  conversion  to  voice 
synthesis/recognition  are  then  rated  by  the  pilots  for 
usefulness  resulting  in  a  pilot  utility  index  for  each.  The 
tasks  are  then  evaluated  by  an  optimization  program  using 
the  performance  specifications  contained  in  Module  Three  to 
determine  what  combination  of  tasks  will  provide  the 
greatest  reduction  in  pilot  workload  without  overloading  the 


voice  channels 


III.  FUTURE  STUDY 


A.  FURTHER  DEVELOPMENT  OF  MODEL'S  INDIVIDUAL  MODULES 

As  stated  in  the  overview  of  the  model,  individual 
modules  representing  distinct  portions  of  the  process  have 
been  used  in  order  to  facilitate  the  refinement  process  for 
this  model. 

Module  One  is  fairly  well  complete  but  the  processing  of 
this  portion  would  be  greatly  enhanced  through  use  of 
automation.  A  computer  program  (possibly  in  Fortran)  could 
be  designed  which  would  ask  the  appropriate  filter  questions 
and  then  generate  the  candidate  Task  List  as  the  final 
output . 

Module  Two  requires  pilot  input  and  could  also  be 
greatly  enhanced  through  the  use  of  a  computer  program.  The 
survey  form  could  be  automatically  generated  from  the 
Candidate  Task  List  prepared  as  part  of  module  one. 

Module  Three  could  be  improved  in  several  ways. 

Although  useful,  better  data  on  reaction  times  could  be 
collected  for  use  in  establishing  performance  specifications 
by  using  actual  F-18  pilots.  Even  if  F-18  pilots  could  not 
be  obtained,  additional  tasks  could  be  tested  on  the 
simulator  in  hopes  of  obtaining  more  accurate  reaction 
times.  In  the  mean  time,  possibilities  for  a  more  reliable 
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method  of  converting  the  values  in  Tables  3,  4,  and  5  could 
be  explored. 

Module  Four  will  work  well  with  any  good  optimization 
package  but  more  study  into  different  weighting  schemes 
could  prove  to  be  valuable.  Both  modules  three  and  four 
would  benefit  from  being  automated  in  a  complete  package 
with  Modules  One  and  Two 

B.  TRAINING  IMPLICATIONS 

The  training  issue  should  be  addressed  for  two  reasons. 
Firstly,  the  Sandry  study  (1982)  showed  a  significant 
reduction  in  reaction  time  for  verbal  tasks  which  were 
converted  to  voice  synthesis/recognition  input/output  modes 
after  only  a  small  amount  of  training  and  practice.  More 
dramatic  results  could  occur  with  additional  practice  by  the 
pilots.  While  pilots  are  practiced  to  the  point  of  an 
automated  response  for  visual  and  manual  actions,  this  is 
not  true  for  audio  and  vocal  ones.  Further,  testing  in  this 
area  could  produce  very  favorable  results  which  would  lead 
to  a  greater  recognition  of  voice  Interaction  in  the  cockpit 
as  a  valuable  asset  for  pilots. 

Secondly,  methods  for  instructing  pilots  in  the  use  of  a 
voice  interactive  system  should  be  explored  so  as  to 
facilitate  the  training  process. 


37 


C .  SYNTAX  DEVELOPMENT 

Selection  of  the  words  (syntax)  to  be  used  with  the 
voice  interactive  system  must  be  made  carefully.  These 
words  must  be  meaningful  to  the  pilot  as  well  as 
distinguishable  from  each  other  by  the  recognition  system. 
Words  which  sound  oo  much  alike  would  lead  to  a  higher 
recognition  error  rate. 

Pilot  input  into  the  selection  process  would  not  only 
result  In  a  superior  vocabulary  but  would  lead  to  greater 
pilot  acceptance. 

D.  SPATIAL  TASKS 

At  this  time  only  verbal  tasks  have  been  considered  as 
candidates  for  voice  synthesis/recognition.  Literature  in 
the  field  as  well  as  the  Sandry  study  (1982)  have  clearly 
demonstrated  the  fact  that  tasks  processed  by  those 
information  processing  channels  which  are  utilized  by 
spacial  tasks  do  not  benefit  by  conversion  to  voice 
input/output  modes. 

However,  the  possibility  does  exist  that  some  spatial 
tasks  could  be  converted  so  that  the  verbal  information 
processing  channels  would  be  used  to  perform  the  task.  If 
this  could  be  accomplished  then  conversion  to  voice  modes 
could  prove  beneficial. 
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For  example,  a  target  tracking  task  could  be  designed 
such  that  the  pilot  could  move  a  cursor  to  a  designated  grid 
by  stating  a  letter -number  combination  rather  that  using  a 
joystick  to  move  the  cursor  to  the  same  area. 

Creative  designing  of  cockpit  tasks  could  greatly 
enhance  the  use  of  the  voice  interactive  system. 


APPENDIX 


EXAMPLE  OF  MODEL  PROCEDURE 

The  purpose  of  this  example  is  to  demonstrate  step  by 
step  the  procedures  described  in  the  model  contained  in 
Chapter  II.  The  treatment  of  the  Included  tasks  has 
deviated  from  fact  in  some  instances  so  that  various  aspects 
of  the  model  could  be  more  fully  illustrated.  The 
Information  for  this  example  has  been  obtained  from  the  Task 
Narratives  contained  in  Appendix  A  of  the  Honeywell  B-52 
Bomber  Study  [North  and  Lea/  1982].  In  this  example  the 
pilot's  job  will  be  analyzed. 

A.  MODULE  ONE 

Since  the  Mission  Scenario  and  the  Task  Narrative  are 
self-explanatory/  this  example  will  begin  with  the  Time- 
based  Activity  Logs  taken  directly  from  the  Honeywell  study 
and  contained  here  as  Tables  6/  7,  8,  and  9.  As  indicated 
by  the  logs,  the  mission  segments  are  Takeoff,  Lowlevel,  Air 
Refueling,  and  Recovery.  These  segments  are  organized  into 
two  missions.  Mission  one  consists  of  Takeoff,  Air 
Refueling,  and  Recovery  while  mission  two  consists  of 
Takeoff,  Lowlevel,  and  Recovery. 
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Table  6.  TAKEOFF  ACTIVITY  LOG 
(Adapted  From  North  and  Lea  (1962)) 
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Table  7.  LOW  LEVEL  ACTIVITY  LOG 
(Adapted  From  North  and  Lea  (1982)) 
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Activity  charts  have  not  been  prepared  £or  this  example 
but  it  should  be  noted  that  single  and  dual  tasking  has  been 
considered  during  this  procedure  as  appropriate. 

The  next  step  in  the  process  is  to  screen  each  task 
listed  on  the  Time-based  Activity  Logs  for  Inclusion  on  the 
Candidate  Task  List.  It  was  decided  that  a  spreadsheet 
approach  would  best  suit  the  procedure  so  Lotus  1-2-3  was 
chosen  for  this  purpose.  Tables  10  and  11  contain  the 
spreadsheets  prepared  in  the  process  of  screening  tasks 
included  in  missions  one  and  two  respectively.  A  step  by 
step  explanation  of  the  process  follows. 

The  first  step  was  to  determine  if  the  task  was 
verbal1 .  If  the  answer  was  affirmative,  "YES"  was  placed 
in  the  appropriate  column.  If  the  answer  was  negative,  the 
task  was  dropped  from  further  consideration  and  "Reject"  was 
entered.  Once  a  task  was  rejected  three  dashes  were  placed 
in  the  remaining  columns  to  indicate  that  this  task  was  no 
longer  under  consideration  for  voice  implementation. 

Next,  each  task  was  processed  through  the  Speech 
Recognition  Filter.  In  response  to  the  question  of  whether 
the  output  mode  was  currently  vocal,  either  "NO"  or  "Reject" 
was  entered.  If  a  "NO"  response  was  entered  the  question  of 
whether  the  task  was  manual  discrete  was  asked.  The  entries 


*Thls  term  is  defined  in  footnote  1. 
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Table  10.  MODULE  ONE  SPREADSHEET — MISSION  ONE 
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Table  11.  MODULE  ONE  SPREADSHEET — MISSION  TWO 
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were  either  "YES"  or  "Reject",  if  the  answer  was  yes  then 
the  question  wao  whether  the  task  occurred  during  time¬ 
sharing.  The  entries  were  then  either  "Accept"  or  "NO".  If 
no,  then  the  task  continued  through  the  filter.  Next,  the 
question  was  asked  about  whether  the  Anthropometry  Index  was 
greater  than  zero.  The  entries  were  either  "Accept"  or 
"NO".  For  "NO"  answers,  the  lest  question  about  whether  the 
task  resulted  in  information  retrieval  was  asked.  The 
entries  were  either  "Accept"  or  "Reject*. 

In  this  example,  no  questions  were  necessary  after  the 
time-sharing  question  although  all  of  the  columns  were 
included  for  completeness.  Lastly,  all  of  the  accepted 
tasks  were  indicated  as  such  by  an  "X"  placed  in  the 
appropriate  column  following  the  Speech  Recognition  Pilter. 

Next,  the  Speech  Generation  Filter  was  applied.  The 
questions  in  this  filter  were  processed  much  the  same  as  for 
the  Speech  Recognition  Filter  and  should  thus  be  self- 
explanatory. 

The  last  question  included  on  the  spreadsheet  dealt  with 
the  technical  feasibility  of  the  tasks  accepted  by  the 
previous  filters.  If  the  task  was  accepted  by  either 
filter,  it  was  screened  for  technical  feasibility.  If  the 
task  was  not  rejected  it  was  listed  on  the  Preliminary 
Candidate  Task  List  shown  in  Table  12. 
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Table  12.  PRELIMINARY  TASK  LIST 
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At  this  point  the  tasks  were  then  grouped  into  types  o£ 
tasks  based  on  whether  they  accessed  the  same  equipment,  had 
the  same  input/output  mode,  required  the  same  number  o£ 
steps  to  complete  and  were  assigned  the  same  anthropometry 
index.  Numbers  with  a  corresponding  key  were  used  to 
identify  like  tasks.  Each  task  which  occurred  more  than 
once  in  either  mission  was  included  on  the  Candidate  Task 
List  (Table  13). 


Table  13.  CANDIDATE  TASK  LIST 

1.  Verbal  Instruction 

2.  Set/Select/Engage/Disengage  (V/M  3)  * 

3.  Checklist 

4.  Set/Select/Engage/Disengage  (V/M  1) 

5.  Set/Select/Engage/Disengage  (V/M  2) 

6.  Adjust  AP/AR  (V/M  3) 

7.  Set/Select/Engage/Disengage  (A/M) 

8.  Adjust  AP/AR  (V/M  1) 

9.  Checks 


B.  MODULE  TWO 

This  module  requires  that  pilots  be  surveyed  and  a  Pilot 
Utility  Index  (PU1)  be  assigned  to  each  of  the  tasks  listed 
m  Table  13.  For  the  purposes  of  this  example,  it  was 
assumed  that  the  pilots  had  been  questioned  and  the  PUl(s) 
listed  in  Table  14  obtained. 


J  The  information  in  parentheses  Indicate  the  input/output 
mode  and  anthropometry  index  of  the  task  listed.  This 
information  is  provided  when  tasks  with  identical  titles  are 
included . 
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Table  14.  PILOT  UTILITY  INDEX 


Task 

PUI 

1. 

Verbal  Instruction 

(V/M  3^° 

4.2 

2. 

Set/Select/Engage/Disengage 

5.2 

3. 

Checklist 

3.5 

4. 

Set/Select/Engage/Disengage 

(V/M  1) 

4.8 

5. 

Set/Select/Engage/Disengage 

(V/M  2) 

4.8 

6. 

Adjust  AP/AR  (V/M  3) 

5.5 

7. 

Set/Select/Engage/Disengage 

(A/M) 

4.8 

8. 

Adjust  AP/AR  (V/M  1) 

5.0 

9. 

Checks 

5.5 

C.  MODULE  THREE 

Module  Three  contains  the  performance  specifications 
developed  for  use  in  completing  the  optimization  in  module 

four.  A  conversion  factor  was  obtained  for  each  task  listed 

‘ ^  \ .  '  ,  .■ , 

on  Table  13  b&;sed  on  the  number  of  steps  the  each  task 

requires. 

D .  MODULE  FOUR  ' 

The  optimization  problem  for  this  example  has  the  same 

formulation  as  described  in  module’  Sour  of  the  model 

description  contained  in  Chapter  II.'  The ,  only  exception  is 

that  the  type  of  task  indicator  is  not  necessary  since  no 

single  ta&ks  remained  for  consideration  after  the  filters 
,  '  ! 
were  applied. 

As  mentioned  in  Chapter  II,  the  LINDO  software  package 
by  LINDO,  Inc  was  used  to  solve  this  problem.  Figure  8 


%he  information  in  parentheses  indicate  the  input/output 
mode  anthropometry  index  of  the  task  listed.  This 
information  Is  provided  when  tasks  with  identical  titles  are 
included. 


51 


contains  the  LINDO  problem  formulation.  For  each  mission,  C 
(the  maximum  total  additional  time  allowed  for  voice 
processing),  was  calculated  by  deciding  the  total  amount  of 
time  which  could  be  spent  in  voice  information  processing 
channels  during  a  mission  and  subtracting  from  it  the  time 
currently  consumed  in  these  channels  by  non-candidate  tasks. 
The  time  was  calculated  by  using  the  conversion  factor 
calculated  above  along  with  the  reaction  times  contained  in 
Table  3.  The  utility  value  Tj^was  calculated  by  summing 
the  reaction  time  savings  (conversion  factor  x  appropriate 
Table  5  entries  x  number  of  times  tasks  performed  during  the 
mission),  pilot  utility  index,  and  the  anthropometry  index. 

Figure  9  contains  the  solution  produced  by  the  LINDO 
package.  Tasks  two  through  eight  were  selected  to  be 
converted  to  the  audio/vocal  mode  while  task  nine  was 
selected  for  conversion  to  the  audio/manual  mode.  Since 
task  one  contains  a  fractional  value,  it  is  dropped  from 
conversion  consideration.  This  combination  of  tasks  are  now 
used  by  the  engineer  to  begin  design  and  testing. 
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Figure  8.  Llndo  Mathematical  Formulation  11 


X j ji.  with  i  =  Candidate  Ta3k 
j  *  Mission  Number 
k  =  Input/Output  Mode 
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